1. Field of the Invention
The present invention relates to image processing apparatuses and methods for detecting, by using images captured by a television camera mounted on a moving object such as an automobile, an obstacle to the moving object.
2. Discussion of the Background
Proposed methods for detecting an obstacle in the vicinity of a vehicle such as an automobile are classified into two types: one using radar and the other using images.
The method using radar often employs laser radar (Japanese Patent Application No. 9-30798) or millimeter wave radar (Japanese Unexamined Patent Application Publication No. 2001-42025). The method using millimeter wave radar has problems in that it cannot distinguish between a wave reflected from the road surface and a wave reflected from a stationary obstacle and that an apparatus implementing the method is expensive. The method using radar has a problem in that it cannot determine whether or not a detected obstacle is on a travel path of the vehicle mounting such a radar system.
In contrast, the method using images detects a vehicle's travel lane (Nakayama, Kubota, Taniguchi, and Onoguchi, “Detection of white line by tracking candidate on back projected image,” Technical Report of Proc. of IEICE, Vol. 101, No. 302, pp. 15-22, 2001) and distinguishes between the road surface and an obstacle that is not on the road surface (Japanese Unexamined Patent Application Publication No. 2001-283204; and Okada, Taniguchi, and Onoguchi, “Obstacle detection by monocular on-vehicle camera using virtual plane tracking method,” Technical Report of Proc. of IEICE, Vol. 101, go. 302, pp. 29-36, 2001). Therefore, the method is capable of detecting an obstacle that is on the vehicle's travel path and that may collide with the vehicle. An apparatus implementing the method is inexpensive.
The method using images is classified into two types: one using a plurality of television cameras and the other using a single television camera.
The method using a plurality of television cameras includes a method of detecting an obstacle by measuring distance on the basis of the principle of triangulation (Japanese Unexamined Patent Application Publication No. 2000-207693) and a method of detecting an object not being on the road surface as an obstacle (Japanese Unexamined Patent Application Publication No. 2000-293693). The method using a plurality of cameras involves calibration of the cameras, and the configuration of an apparatus implementing the method is complicated. The apparatus including the plural cameras is more expensive than an apparatus including a single camera.
The method using a single camera includes a method of detecting a horizontal edge line as a preceding vehicle's ground line (Japanese Unexamined Patent Application Publication No. 7-280517 and Japanese Unexamined Patent Application Publication No. 7-28975). This method may make a detection error since the detected edge line may be the seam of an asphalt road or the border of road markers painted on a road surface.
In S. Carlsson and J. O. Eklundh, “Object detection using model based prediction and motion parallax,” Proc. of ECCV, pp. 297-306, 1990, the motion of the road surface is estimated using the motion of a set of points in an image, and an area that moves differently from the motion of the road surface is detected as an obstacle area.
As in this technique, a technique for estimating the motion of the road surface requires that the road surface occupy a large image region relative to obstacles and that the road surface must have a clear texture (a road marker, e.g., a lane marker or a pedestrian crossing). Such conditions may not often be satisfed by the actual road scenes.
In Ko and Uchimura, “Detection and simultaneous tracking of a plurality of moving bodies in changing background,” IEEJ Transactions on Industry Applications, Vol. 120, No. 10, a vehicle's traveling direction is estimated from the motion of a set of points in an image, and an area moving in a direction differing from the moving direction of the background is detected as an obstacle. On the basis of the geometrical relationship between the television camera and the road surface, a change in distance from the vehicle to the obstacle is computed. A moving object that moves at a speed differing from the vehicle's relative speed with respect to the road surface is also detected as an obstacle. This technique has a problem in that it cannot detect a stationary object on the road.
In Zhang, et al., “Obstacle detection based on qualitive and quantitative 3D reconstruction,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 19, No. 1, pp. 15-26, 1997, a case is described in which some of the features in an image are on the road surface and the other features belong to an obstacle. In such a case, on the basis of the fact that the motion of the features cannot be described in terms of the motion in a three-dimensional plane, an obstacle is detected when the motion of a set of features cannot be described in terms of planar motion. This technique needs a sufficient number of features belonging to both the road surface and the obstacle. In the actual road scene, there may be almost no features on the road surface.
Since misestimating correspondences of features over images causes a detection error, this technique is difficult to use in the actual environment.
In contrast, in Okada, et al., “Obstacle detection by monocular on-vehicle camera using virtual plane tracking method,” two virtual planes whose directions are respectively identical to that of the road surface and an obstacle are arranged in a scene for an image region, and are tracked in an image sequence. A virtual plane in which the tracking result best matches the current image is selected, thus distinguishing between the obstacle and the road surface.
This technique also utilizes motion parallax between the road surface and the obstacle. Since this technique requires no features on the road surface, this technique is easily applicable to the actual road scene. Since this technique does not track the feature points based on local information, but tracks the planes, stability is improved. On the other hand, this technique involves high computational cost because it tracks many planes.
In order to realize the method based on motion parallax between the road surface and the obstacle under the actual environment, the video-rate image processing is necessary, and the detection result must be accurate. The motion parallax needs to be reliably computed with low computational cost.